Method for converting acoustic signal into haptic signal, and haptic device using same

ABSTRACT

A haptic device according to one embodiment can comprise: a database unit for storing acoustic information or receiving the acoustic information from an external device; a control unit for converting the acoustic information into an electrical signal according to a predetermined pattern; a driving unit for generating a motion signal on the basis of the electrical signal; and a transfer unit for transferring a patterned tactile signal to a user by means of the motion signal.

TECHNICAL FIELD

Embodiments relate to a method of converting an acoustic signal into atactile signal and a haptic device using the same.

BACKGROUND

A haptic device provides a tactile sense to a user by generating avibration, a force, or an impulse in a digital device. That is, thehaptic device provides a vibration, a motion, or a force to the userwhen the user controls an input device (for example, a joystick, amouse, a keyboard, or a touch screen) of the digital device such as agame console, a mobile phone, or a computer. Thus, the haptic devicetransmits realistic information to the user, like computer virtualexperience.

In general, haptic devices for haptic technology include an inertialactuator, a piezoelectric actuator, and an electro-active polymeractuator (EAP).

The inertial actuator includes an eccentric rotation motor (ERM) thatvibrates using an eccentric force generated by a weight body connectedto a magnetic circuit, and a linear resonant actuator (LRA) thatmaximizes an intensity of vibration using a resonant frequency generatedby a weight body connected to a magnetic circuit and an elastic spring.

The piezoelectric actuator is a device that is driven in a form of a baror a disk using an elastic body, around a piezoelectric device with ashape that instantaneously deforms by an electric field.

In relation to the piezoelectric actuator, among the existing hapticdevices, there were published Korean Patent Publication No. 10-1603957(entitled Piezoelectric Actuator, Piezoelectric Vibration Apparatus, andPortable Terminal), and Korean Patent Application Publication No.10-2011-0118584 (entitled Transparent Composite Piezoelectric CombinedTouch Sensor and Haptic Actuator).

The EAP is a device that is driven by providing repeated motions usingan electro-active polymer film attached to a mass body, on the mainprinciple that a shape thereof deforms by a functional group of apolymer backbone having a specific mechanism by external electric power.

Further, in addition to the above haptic devices, haptic devices usingshape-memory alloys, electrostatic forces, or ultrasonic waves are beingdeveloped.

At an early stage of development, the haptic device was mainly appliedto aircraft and fighter aircraft simulations, virtual reality experiencefilms, and games. With the release of a touch screen mobile phone towhich haptic technology is applied after the mid 2000's, the haptictechnology has emerged and become familiar to individual users.

As described above, the haptic device is used in various electronicdevices such as game consoles. Use of the haptic device is increasing inresponse to a growing user demand for accessing media using a complexmethod such as a tactile method or an olfactory method in addition to anaudiovisual method.

In general, an existing haptic feedback providing method operates thehaptic device in response to an event generated when a user controls adigital device, or an event generated in an application. That is, thehaptic device is triggered by a predetermined event generated when theuser interacts through a user interface of the digital device, or theevent generated in the application (for example, an alarm). Like this,in general, an event-driven haptic feedback providing method thatoutputs a predefined predetermined haptic pattern in response to thegenerated event is used.

Another haptic feedback providing method provides a haptic feedback bycontinuously converting output audio data into data for haptic output.In this example, the output audio data is converted into the haptic datausing an analog signal scheme or a fast Fourier transform (FFT) filterscheme.

The analog signal scheme drives a haptic actuator using an analog signalgenerated when outputting audio data as an input. The analog signalscheme has an overly fast response rate, is easily implemented ashardware, and is more effective for a case in which a driving frequencyrange of the actuator is varied.

SUMMARY

An aspect provides technology for providing a tactile sense appropriatefor an acoustic effect output from various contents such as images,music, and games.

Another aspect provides technology for providing a more rhythmic andhigher-dimensional tactile sense, rather than a simple vibration, bydistinguishing a loudness and a pitch.

Still another aspect provides technology for efficiently patterning, astactile senses, records such as a score in which an acousticcharacteristic is written.

According to an aspect, there is provided a haptic device including adatabase configured to store acoustic information or receive theacoustic information from an external device, a controller configured toconvert the acoustic information into an electrical signal correspondingto a predetermined pattern, and a tactile actuator configured to providea user with a patterned tactile signal. The tactile actuator may includea driver configured to generate a motion signal based on the electricalsignal, and a transmitter configured to transmit the patterned tactilesignal to the user using the motion signal.

The acoustic information may include at least one of a note havingduration information, pitch information, and loudness information, and arest having duration information.

The electrical signal corresponding to the predetermined pattern mayinclude a voltage magnitude that increases in proportion to the pitchinformation of the note, a voltage apply time that increases inproportion to the duration information of the note, and a waiting timearranged after the voltage apply time.

The voltage magnitude may have a predetermined value corresponding to anote having a different pitch.

The duration information of the note may be equal to a sum of thevoltage apply time and the waiting time.

The acoustic information may further include a dynamic marking thatchanges the loudness information of the note, and the controller may beconfigured to adjust the voltage magnitude or the voltage apply timebased on an indication indicated by the dynamic marking.

The controller may be configured to increase the voltage magnitudeand/or the voltage apply time compared to the waiting time at apredetermined rate when the dynamic marking is a forte-type marking, andthe controller may be configured to decrease the voltage magnitudeand/or the voltage apply time compared to the waiting time at thepredetermined rate when the dynamic marking is a piano-type marking.

The acoustic information may further include a slur that links notes ofdifferent pitches, and the controller may be configured to adjust awaiting time between the notes linked by the slur to “0” seconds.

The acoustic information may further include a tie that links notes ofthe same pitch, and the controller may be configured to add a durationof a following note to a waiting time between the notes linked by thetie.

The acoustic information may further include a bar indicating that apredetermined section is to be repeated, and the controller may beconfigured to adjust the voltage such that voltages corresponding tonotes in a section indicated by the bar may be repeated.

The driver may include a housing having an accommodation space therein,a vibrator disposed in the accommodation space, an elastic memberconfigured to connect the housing and the vibrator such that thevibrator vibrates with respect to the housing, and a coil configured toform a magnetic field to drive the vibrator, the transmitter may bedisposed to cover the accommodation space, a mass of the vibrator may bebelow 2 grams (g), an elasticity coefficient of the elastic member maybe below 2.021 newtons per millimeter (N/mm), and a resonant frequencyof the tactile actuator may be below 160 hertz (Hz).

The controller may be configured to determine one of a plurality ofpredetermined driving modes based on the acoustic information, and theelectrical signal may have a frequency corresponding to the determineddriving mode.

The plurality of driving modes may include a general vibration mode, atapping mode, and a rolling mode. The controller may be configured toapply, to the coil, a sine wave electrical signal of a frequency below160 Hz when the determined driving mode is the general vibration mode,the controller may be configured to apply, to the coil, a square wave orpulse wave electrical signal of a frequency below 60 Hz, which is lowerthan the frequency of the electrical signal applied in the generalvibration mode, when the determined driving mode is the tapping mode,and the controller may be configured to apply, to the coil, a sine waveor pulse wave electrical signal of a frequency lower than the frequencyof the electrical signal applied in the general vibration mode andhigher than the frequency of the electrical signal applied in thetapping mode when the determined driving mode is the rolling mode.

The acoustic information may include information related to whether adynamic marking changing the loudness information of the note is presentand a type of the dynamic marking, and the controller may be configuredto determine the driving mode based on whether the dynamic marking ispresent and the type of the dynamic marking.

The acoustic information may include information related to a slur thatlinks notes of different pitches, and the controller may be configuredto determine the driving mode to be the rolling mode in a section inwhich notes are linked by the slur.

According to an aspect, there is provided a method of converting anacoustic signal into a tactile signal using a haptic device, the methodincluding inputting stored acoustic information or acoustic informationreceived from an external device, interpreting details of the acousticinformation based on a five-line staff, converting the acousticinformation into an electrical signal corresponding to a predeterminedpattern, generating a motion signal based on the electrical signal, andtransmitting a patterned tactile signal to a user using the motionsignal.

The interpreting may include determining an arrangement of at least onenote or rest, verifying the duration information, the pitch information,and the loudness information of the note, and verifying the durationinformation of the rest.

The converting may include matching a predetermined correspondingvoltage magnitude to each note, and adjusting the voltage magnitude ofthe electrical signal based on the matched voltage magnitude.

The generating may include generating the motion signal having anamplitude and a frequency corresponding to the pattern of the electricalsignal.

The transmitting may include transmitting the tactile signal to afingertip of the user.

According to an embodiment, it is possible to efficiently pattern, astactile senses, records such as a score in which an acousticcharacteristic is written.

According to an embodiment, it is possible to provide a more rhythmicand higher-dimensional tactile sense, rather than a simple vibration, bydistinguishing a loudness and a pitch.

According to an embodiment, it is possible to provide a tactile senseeffective for an acoustic effect output from various contents such asimages, music, and games.

According to an embodiment, it is possible to increase a user experienceof a content user such that the user may receive a great sensitivity.

According to an embodiment, it is possible to provide tactile sensesmore efficiently than the existing technology, in a frequency rangebelow 160 hertz (Hz), of a frequency range that may be sensed by a humanbody.

BRIEF DESCRIPTION OF DRAWINGS

The accompanying drawings illustrate preferred embodiments of thepresent invention, and are provided together with the detaileddescription for better understanding of the technical idea of thepresent invention. Therefore, the present invention should not beconstrued as being limited to the embodiments set forth in the drawings.

FIG. 1 is a view illustrating a haptic device according to anembodiment.

FIG. 2 is a table illustrating notes and rests used in a five-linestaff.

FIG. 3 is a table illustrating time signatures used in a five-linestaff.

FIG. 4 illustrates a tempo, an articulation, and dynamic markings usedin a five-line staff.

FIG. 5 is a score illustrating bars used in a five-line staff.

FIG. 6 illustrates pitches, a slur, and a tie used in a five-line staff.

FIG. 7 is a graph illustrating a process of converting acousticinformation into an electrical signal.

FIG. 8 is a table illustrating an example of changing an intensity of anelectrical signal based on a type of a dynamic marking.

FIG. 9 illustrates an example of a five-line staff with an indication ofa tempo.

FIG. 10 is a flowchart illustrating a method of converting acousticinformation into a tactile signal according to an embodiment.

FIG. 11 is a flowchart illustrating an interpreting operation of amethod of converting acoustic information into a tactile signalaccording to an embodiment.

FIG. 12 is a flowchart illustrating a converting operation of a methodof converting acoustic information into a tactile signal according to anembodiment.

FIG. 13 is a flowchart illustrating a generating operation and atransmitting operation of a method of converting acoustic informationinto a tactile signal according to an embodiment.

FIG. 14 illustrates an inside of a tactile actuator according to anembodiment.

FIG. 15 illustrates an elastic member according to an embodiment.

FIG. 16 illustrates an elastic member according to another embodiment.

FIG. 17 is a block diagram of a tactile actuator according to anembodiment.

FIG. 18 is a graph conceptually illustrating a driving region withrespect to a frequency, in a tactile actuator according to anembodiment.

FIG. 19 is a graph illustrating a relationship between an actuallymeasured frequency and an acceleration, in a tactile actuator accordingto an embodiment.

FIG. 20 is a graph illustrating a relationship between a measuredfrequency and an acceleration when a square wave electrical signalhaving a low frequency is applied, in a tactile actuator according to anembodiment.

FIG. 21 is a graph illustrating a relationship between a measuredfrequency and an acceleration when a sine wave electrical signal havinga low frequency is applied, in a tactile actuator according to anembodiment.

FIG. 22 illustrates a control method for a tactile actuator according toan embodiment.

FIG. 23 illustrates an example of an operation of a tactile actuatoraccording to an embodiment.

FIG. 24 illustrates another example of an operation of a tactileactuator according to an embodiment.

FIG. 25 is a graph illustrating a change in an acceleration with respectto a change in an intensity of a square wave input electrical signal of5 hertz (Hz), in tactile actuators having different resonantfrequencies.

FIG. 26 is a graph illustrating a change in an acceleration with respectto a change in a frequency of a square wave input electrical signal of90 milliamperes (mA), in tactile actuators having different resonantfrequencies.

FIG. 27 illustrates waveforms of a vibrator exhibited in response to achange in a square wave electrical signal input into a tactile actuatorhaving a resonant frequency characteristic of 80 Hz.

FIG. 28 illustrates waveforms of a vibrator exhibited in response to achange in a square wave electrical signal input into a tactile actuatorhaving a resonant frequency characteristic of 120 Hz.

FIG. 29 illustrates waveforms of a vibrator exhibited in response to achange in a square wave electrical signal input into a tactile actuatorhaving a resonant frequency characteristic of 160 Hz.

FIG. 30 illustrates waveforms of a vibrator exhibited in response to achange in a square wave electrical signal input into a tactile actuatorhaving a resonant frequency characteristic of 180 Hz.

FIG. 31 is a graph illustrating threshold frequencies of tapping andvibration when a square wave electrical signal is applied, in tactileactuators having different resonant frequencies.

FIG. 32 illustrates waveforms of a vibrator exhibited in response to achange in a pulse wave electrical signal input into a tactile actuatorhaving a resonant frequency characteristic of 80 Hz.

FIG. 33 illustrates waveforms of a vibrator exhibited in response to achange in a pulse wave electrical signal input into a tactile actuatorhaving a resonant frequency characteristic of 120 Hz.

FIG. 34 illustrates waveforms of a vibrator exhibited in response to achange in a pulse wave electrical signal input into a tactile actuatorhaving a resonant frequency characteristic of 160 Hz.

FIG. 35 illustrates waveforms of a vibrator exhibited in response to achange in a pulse wave electrical signal input into a tactile actuatorhaving a resonant frequency characteristic of 180 Hz.

FIG. 36 is a graph illustrating threshold frequencies of tapping andvibration when a pulse wave electrical signal is applied, in tactileactuators having different resonant frequencies.

FIG. 37 illustrates waveforms of a vibrator exhibited in response to achange in a sine wave electrical signal input into a tactile actuatorhaving a resonant frequency characteristic of 80 Hz.

FIG. 38 illustrates waveforms of a vibrator exhibited in response to achange in a sine wave electrical signal input into a tactile actuatorhaving a resonant frequency characteristic of 120 Hz.

FIG. 39 illustrates waveforms of a vibrator exhibited in response to achange in a sine wave electrical signal input into a tactile actuatorhaving a resonant frequency characteristic of 160 Hz.

FIG. 40 illustrates waveforms of a vibrator exhibited in response to achange in a sine wave electrical signal input into a tactile actuatorhaving a resonant frequency characteristic of 180 Hz.

FIG. 41 is a graph illustrating threshold frequencies of rolling andvibration when a sine wave electrical signal is applied, in tactileactuators having different resonant frequencies.

FIG. 42 illustrates a control method for a tactile actuator according toanother embodiment.

FIG. 43 illustrates driving modes in which a tactile actuator operatesbased on types of dynamic markings according to an embodiment.

DETAILED DESCRIPTION

Hereinafter, embodiments of the present disclosure will be describedwith reference to the accompanying drawings. Regarding the referencenumerals assigned to the components in the drawings, it should be notedthat the same components will be designated by the same referencenumerals, wherever possible, even though they are shown in differentdrawings. Also, in the description of the embodiments, detaileddescription of well-known related structures or functions will beomitted when it is deemed that such description will cause ambiguousinterpretation of the present disclosure.

Also, in the description of the components, terms such as first, second,A, B, (a), (b) or the like may be used herein when describing componentsof the present disclosure. Each of these terms is not used to define anessence, order or sequence of a corresponding component but used merelyto distinguish the corresponding component from other component(s). Itshould be noted that if it is described in the specification that onecomponent is “connected,” “coupled” or “joined” to another component,the former may be directly “connected,” “coupled,” and “joined” to thelatter or “connected”, “coupled”, and “joined” to the latter via anothercomponent.

FIG. 1 is a view illustrating a haptic device 1 according to anembodiment.

Referring to FIG. 1, the haptic device 1 may include a database D 10that stores acoustic information or receives the acoustic informationfrom an external device S, a controller C 20 that converts the acousticinformation into an electrical signal corresponding to a predeterminedpattern, and a tactile actuator 200 that receives the electrical signalfrom the controller 20 and transmits various tactile signals to a user.

The tactile actuator 200 may include a driver A 30 that generates amotion signal based on the electrical signal, and a transmitter H 40that is physically connected to the driver 30 to transmit a patternedtactile signal to the user using the motion signal. An exemplaryconfiguration of the tactile actuator 200 will be described further withreference to FIGS. 14 through 16.

The external device S may include all devices that may store andtransmit the acoustic information, and may be, for example, a typicallyused mobile device, a universal serial bus (USB) storage medium, or asecure digital (SD) card. For example, the external device S may beconnected directly to the database 10, or may transmit the acousticinformation to the controller 20 and/or the database 10 through acommunicator 50 of FIG. 17 that receives the acoustic information in awired or wireless manner using the Internet.

The acoustic information may include a text or an image includinginformation related to notes, rests, time signatures, tempos,articulations, dynamic markings, and various signs modifying the samethat may be written in a score such as a five-line staff, rather than asource in a form of a sound file such as moving picture experts group(MPEG) audio layer-3 (MP3) which is generally used. The acousticinformation may be optimized for the haptic device 1, and thusinformation may be stored or transmitted and received with a muchsmaller capacity, when compared to a case of using a sound file.

Music notation (score writing) is the most well known music writingsystem. Each culture has a unique form of music notation. In general,different scores may be notated in conjunction with each other.Hereinafter, a case in which the haptic device 1 uses a five-line staffwhich is the most public score among various scores will be described.However, examples may also be applied to scores other than the five-linestaff. The haptic device 1 may convert each piece of acousticinformation into an electrical signal to be used to provide a tactilesignal.

The controller 20 may control a motion of the driver 30 based on theacoustic information. The controller 20 may convert a composition of thescore into an electrical signal to be used to drive the driver 30, andthe driver 30 may output the electrical signal as a motion signalincluding a real motion.

The transmitter 40 may be in direct contact with a skin of a hand oranother body part of a user, and transmit, to the user, a tactile sensecorresponding to a sound being played through a cycle or an intensity ofa tactile signal. For example, the transmitter 40 may be a cover or acase that encloses the driver 30. In another example, the transmitter 40may be a wearable device such as a glove or a watch, or a haptic displaycapable of a three-axial transitional motion and a three-axialrotational motion.

FIGS. 2 through 6 illustrate various signs used in a five-line staff.

FIG. 2 is a table illustrating notes and rests used in a five-linestaff.

A note is a symbol that represents a duration and a pitch of a sound.The duration of the sound may be represented by a type of the note, andthe pitch of the sound may be represented by a position of the note onfive lines.

A rest is a symbol that represents an interval of silence in a piece ofmusic and a duration thereof. The rest does not have a pitch, and thusis generally written at a predetermined position irrespective of aheight on five lines. Durations of rests correspond to durations ofcorresponding notes having the same names.

With respect to a duration of a sound, “whole” refers to one beat, andhalf or quarter refers to a ½ beat or a ¼ beat. Thus, a note or a reston a lower side of the table may have a shorter duration.

FIG. 3 is a table illustrating time signatures used in a five-linestaff.

A time signature is a fractional number or a sign (symbol) that followsa clef or a key signature on a right side thereof in a score. Adenominator of the fractional number indicates a type of a note being aunit of one beat, and a numerator of the fractional number indicates thenumber of beats per measure.

For example, time signatures are expressed by fractional numbers such as4/4, 3/4, 2/4, and 6/8, where denominators thereof each indicate a unitnote, and numerators thereof each indicate the number of unit notes permeasure. For example, the 4/4 time indicates a time in which a quarternote is one beat, and 4 beats are included in one measure.

FIG. 4 illustrates a tempo, an articulation, and dynamic markings usedin a five-line staff.

A tempo indicates a speed of progression of a piece of music or aregulation thereof, and a pace to play the piece of music.

Further, the tempo indicates how to use a metronome, determines a unitnote, and indicates the number of unit notes to be played per minute.For example, “

=120” indicates that a ⅛ beat appears 120 times per minute. In thisexample, a piece of music is to be played at a tempo where a total of 15beats appears per minute.

Articulations refer to various methods of musically articulating andlinking notes according to a theme of music. The articulations are usedtogether with phrasing. The phrasing divides a melody by a phrase of apredetermined size, whereas an articulation divides the melody by asmaller unit than the phrasing.

The articulations include legato, non-legato, staccato, and portato.Legato indicates that notes are to be played to be linked withoutseparation. Non-legato indicates that notes are to be played fordurations shorter than values thereof with a temporal pausetherebetween. Staccato indicates that notes are to be played shortseparately with about half durations thereof. Portato indicates thatnotes are to be played by sufficiently increasing values thereof throughtonguing of each note.

Dynamic markings are markings indicating a vibration in loudness betweennotes in a portion of a piece of music or in the whole piece of music,and are also referred to as dynamic signs. The dynamic markings aremarkings used to more minutely and clearly indicate an expression or acharacteristic of the piece of music. The dynamic markings are writtenat the beginning or in the middle of the piece of music to indicate orchange loudnesses of notes such as overall dynamics or partial dynamicsof the piece of music.

In general, the dynamic markings include pianissimo (pp: very soft),piano (p: soft), mezzo-piano (mp: moderately soft), mezzo-forte (mf:moderately loud), forte (f: loud), fortissimo (ff: very loud), crescendo(cresc: gradually loud), decrescendo (decresc: gradually soft),sforzando (sf: forcefully loud), and fortepiano (fp: loud followed byimmediately soft).

FIG. 5 is a score illustrating bars used in a five-line staff.

For example, a repeat sign designates a section to be repeated on ascore. The repeat sign includes a sign of two dots beside a double barin a direction to repeat the section, and a sign using an indication ofD.S. or D.C. or a text of bis or ter.

FIG. 6 illustrates pitches, a slur, and a tie used in a five-line staff.

A pitch indicates how high and low a sound is. When a note is positionedat a higher position on a five-line staff, a sound of the note has ahigher pitch.

A slur is an arc attached above or below two or more notes of differentpitches. The slur indicates that notes that the slur embraces are to beplayed softly and smoothly (with legato articulation).

A tie is an arc linking two notes of the same pitch at a position aboveor below the notes. The two notes linked by the tie are played as onecontinuous sound without separation. The tie links notes in the samemeasure, and also links notes in different measures across a bar.

FIG. 7 is a graph illustrating a process of converting acousticinformation into an electrical signal.

The controller 20 of the haptic device 1 may convert acousticinformation into an electrical signal. Elements to define the electricalsignal are as follows.

a. Voltage magnitude

b. Voltage polarity (+, −)

c. Voltage apply time

d. Waiting time

Acoustic information constituting a score may be converted into anelectrical signal to be used to operate a tactile device using thefollowing method.

During the voltage apply time, the electrical signal may have a constantvoltage Va, Vb of a predetermined magnitude. During the waiting time, avoltage of the electrical signal may be “0”.

A duration of a sound of a single note in a score is equal to a sum ofthe voltage apply time and the waiting time of the electrical signal.Meanwhile, the voltage apply time and the waiting time may be determinedby a preset rate.

The duration of the sound of the single note in the score may becalculated based on a time signature written in the score, a tempo, andthe number of beats indicated by the note. The duration may be appliedas the sum of the voltage apply time and the waiting time of theelectrical signal.

In a case in which a rest is present in the score, the number of beatsindicated by the rest may be added to a waiting time of an immediatelyprevious note such that the voltage magnitude of the electrical signalmay be continuously maintained to be “0”.

The voltage of the electrical signal to be applied by the controller 20may be determined based on the number of pitches of notes written in ascore received by the database 10, within a voltage range in which thedriver 30 is properly driven.

For example, the haptic device 1 having a driving voltage range between1 volt (V) and 8 V may be used. In a case of converting, into a tactilepattern, a score including “CDEFGABC”, in which the highest sound is “C”of the second octave and the lowest sound is “C” of the first octave,the driving range of 1V to 8V may be divided by 8 equal parts based onthe respective pitches and distributed thereto. For example, thecontroller 20 may determine a voltage of the electrical signalcorresponding to “C” of the second octave, which is the lowest note, tobe 1 V, sequentially determine a voltage corresponding to “D” to be 2 V,and finally determine a voltage corresponding to “C” of the third octaveto be 8 V. Voltage magnitudes of the electrical signal corresponding topitches of sounds indicated by the notes are exemplarily shown in thetable of FIG. 8.

Meanwhile, in a case in which the number of cases of driving voltages ofthe driver 30 is less than the number of sounds from the lowest sound tothe highest sound of the notes included in the score, the controller 20may set a region by dividing the total number of the sounds from thelowest sound to the highest sound by the number of cases of the drivingvoltages, and drive a sound with a different voltage for each region.That is, the controller 20 may drive a plurality of sounds belonging tothe same region with the same voltage.

As shown in FIG. 6, in a case in which notes having different pitchesare linked by a slur, the voltage magnitude of the electrical signal maychange to distinguish a pitch of each sound, a voltage apply time of apreceding note may be set to be equal to a total duration of thecorresponding note, and a waiting time may be set to 0 seconds (s).

In a case in which notes having the same pitch are linked by a tie, awaiting time of a preceding note may be newly set by adding a totalduration of a following note to the original waiting time of thepreceding note, and information related to the following note may beignored. By the above scheme, the acoustic information may be moresimplified, whereby an amount of information written in a score may bereduced further and thus, the acoustic information may be regenerated asinformation optimized for the haptic device 1.

The controller 20 may interpret meanings of bars such as an end sign anda repeat sign, or abbreviations such as Di capo and Dal Segno that arewritten in a score, and generate repeated patterns of the electricalsignal based on instructions indicated by the corresponding signs.

Further, the controller 20 may not consider a key signature and amodulation of the score, and may determine a magnitude of voltage to beapplied based on a pitch of a note in its original position.

FIG. 8 is a table illustrating an example of changing an electricalsignal based on a type of a dynamic marking. Types of dynamic markingsare shown in FIG. 4.

The controller 20 may interpret a meaning of a dynamic marking, andadjust a magnitude of voltage to be applied or a voltage apply timebased on a type of the dynamic marking.

For example, a haptic device having a driving voltage range between 1 Vand 8 V may be used. A mezzo-piano score including “CDEFGABC”, in whichthe highest sound is “C” of the second octave and the lowest sound is“C” of the first octave, has a softer mood than a moderato (moderateloudness) score, and thus a voltage to be applied thereto may be set tobe lower.

In detail, a pitch-voltage relationship may be set as four stages of(CD)-(EF)-(GA)-(BC), and the voltage to be applied may be distributed tothe stages from 1 V to 4 V such that the electrical signal may have arelatively weak overall magnitude of the voltage to be applied whencompared to the moderato score.

In another example, a haptic device having a driving voltage rangebetween 1 V and 8 V may be used. A mezzo-forte score including“CDEFGABC”, in which the highest sound is “C” of the second octave andthe lowest sound is “C” of the first octave, has a louder mood than amoderato (moderate loudness) score, and thus an electrical signal may beadjusted to have a relatively longer voltage apply time. That is, in acase in which a dynamic marking is a forte-type marking, informationrelated to the dynamic marking may be reflected by increasing a voltageapply time and decreasing a waiting time. In other words, by increasingor decreasing the voltage apply time compared to the waiting time forall notes, whether dynamics change may be sensed through a tactilesense.

Example 1

Hereinafter, a method of generating a tactile pattern will be suggestedin detail.

FIG. 9 illustrates an example of a five-line staff with an indication ofa tempo.

From a score of FIG. 9, the controller 20 may verify a time signature of4/4 and a tempo indicating that 250 quarter notes are to be played, andcalculate a duration of a sound corresponding to a single quarter notethrough the time signature and the tempo. The calculated duration may be250 milliseconds (ms) (that is, ¼ s).

Based on the calculated duration, the controller 20 may set durations ofan eighth note, a sixteenth note, and a quarter rest used in the score,and set a voltage apply time and a waiting time of an electrical signalcorresponding to each note based on a preset rate.

Further, the electrical signal may have a pattern including a total of 8measures in which 4 measures written in the score are repeated two timesbased on a repeat sign.

In a case in which a tie exists between notes, for example, in a case inwhich two eighth notes are linked by a tie, a waiting time of apreceding note may be newly set by adding a total duration, that is, 125ms, of a following note to the original waiting time of the precedingnote, and information related to the following note may be ignored,whereby the electrical signal may be patterned.

In a case in which a rest is placed in the score, for example, in a casein which a quarter rest comes immediately after a quarter note, awaiting time of the quarter note may be newly set by adding 250 mscorresponding to a duration of the rest to the original waiting time ofthe quarter note previous to the quarter rest, and information relatedto the quarter rest may be ignored, whereby the electrical signal may bepatterned.

In a case in which scores for various musical instruments are providedfor a solo or an ensemble in a single piece of music for an orchestra oran opera, the haptic device 1 may divide a melody written in each scoreby each measure and combine the same in freedom to pattern theelectrical signal, thereby patterning a tactile sense.

FIG. 10 is a flowchart illustrating a method 100 of converting anacoustic signal into a tactile signal according to an embodiment.

The method 100 of converting an acoustic signal into a tactile signalmay include operation 110 of inputting stored acoustic information oracoustic information received from an external device, operation 120 ofinterpreting details of the acoustic information based on a five-linestaff, operation 130 of converting the acoustic information into anelectrical signal corresponding to a predetermined pattern, operation140 of generating a motion signal based on the electrical signal, andoperation 150 of transmitting a patterned tactile signal to a user usingthe motion signal.

The method 100 may use the haptic device 1 described above. Thepatterned tactile signal that the user may finally receive may begenerated based on the motion signal generated in response to a motionof the driver 30 of the tactile actuator 200, and the motion signal maybe controlled based on the patterned electrical signal. Thus, the method100 may produce a patterned electrical signal by interpreting notes,rests, and various signs constituting a score, and transmit a tactilesignal that the user may feel in reality in a patterned form based onthe patterned electrical signal.

FIG. 11 is a flowchart illustrating operation 120 of the method 100 ofconverting an acoustic signal into a tactile signal according to anembodiment.

Operation 120 may include operation 121 of determining an arraignment ofat least one note or rest, operation 122 of verifying durationinformation, pitch information, and loudness information of the note,and operation 123 of verifying duration information of the rest.

The duration information of the note may be interpreted based on a timesignature and a duration of a note written in a score. The durationinformation of the note may be equal to a sum of a voltage apply timeand a waiting time. The electrical signal may be set to have a voltagemagnitude of a predetermined magnitude not being 0 V during the voltageapply time only, and set to have a voltage magnitude of 0 V during thewaiting time.

Similar to the note, the duration information of the rest may beinterpreted based on the time signature and the duration of the notewritten in the score. For the duration of the rest, the voltagemagnitude may be set to 0 V, similar to the waiting time of the note.

FIG. 12 is a flowchart illustrating operation 130 of the method 100 ofconverting an acoustic signal into a tactile signal according to anembodiment.

Operation 130 may include operation 131 of matching a predeterminedcorresponding voltage magnitude to each note, and operation 132 ofadjusting the voltage magnitude of the electrical signal based on thematched voltage magnitude.

After a range of all sounds of notes arranged in a score is verifiedfirst, a voltage magnitude to play each sound may be matched. Differentvoltage magnitudes may be matched to all the sounds, or a portion ofadjacent sounds may be tied and the same voltage may be matched thereto.A scheme of tying sounds and matching may vary depending on a dynamicmarking written in the score.

A voltage magnitude may be adjusted based on a pitch of a soundcorresponding to a note written in the score, and a voltage value may beeither positive or negative. For example, based on a predeterminedoctave, the voltage magnitude may be adjusted to be a minus voltagemagnitude with respect to a note of a lower octave, and adjusted to be aplus voltage magnitude with respect to a note of a higher octave.

FIG. 13 is a flowchart illustrating operation 140 and operation 150 ofthe method 100 of converting an acoustic signal into a tactile signalaccording to an embodiment.

A motion signal used to generate a tactile signal that a user may feelmay include, for example, a translational motion, a vibrational motion,and a rotational motion. Hereinafter, a case in which a motion signal isa vibrational signal will be described in detail.

Operation 140 may include operation 141 of generating the motion signalhaving an amplitude and a frequency corresponding to the pattern of theelectrical signal. The vibrational signal may be controlled based on acharacteristic of the electrical signal.

For example, in a case in which a sound of a note has a high pitch andthus the voltage of the electrical signal is great, an amplitude of thevibrational signal may increase. In a case in which the sound of thenote has a long duration, a frequency of the vibration may increase.During a waiting time of the note or for a duration of a rest, thevibrational signal may be paused.

Operation 150 may include operation 151 of transmitting the vibrationalsignal to a fingertip of the user. The vibrational signal may betransmitted through a display device or a wearable device to be worn ona hand of the user.

As described above, when the characteristic of the vibrational signalchanges, the user may feel transmission of a different sound with thefingertip.

With reference to FIGS. 14 through 43, a configuration of the tactileactuator 200 generating various tactile patterns (general vibrationmode, tapping mode, rolling mode), experiment data, and a method andutilization of converting a music signal into various tactile signalsusing the same will be described.

FIG. 14 illustrates an inside of the tactile actuator 200 according toan embodiment, FIG. 15 illustrates an elastic member 204 according to anembodiment, FIG. 16 illustrates an elastic member 214 according toanother embodiment, and FIG. 17 is a block diagram of the tactileactuator 200 according to an embodiment.

Referring to FIGS. 14 through 17, the tactile actuator 200 may includethe driver 30 and the transmitter 40, and the driver 30 may include ahousing 201, a vibrator 203, the elastic member 204, and a coil 205.

The housing 201 may include, for example, an accommodation space with anopened top. Although the housing 201 is illustrated as a box shape, theshape of the housing 201 is not limited thereto.

The transmitter 40 may cover at least a portion of the accommodationspace. An edge portion of the transmitter 40 may be fixed to a side wallof the housing 201. Through a body of a user in direct or indirectcontact with the transmitter 40, vibration generated by the vibrator 203may be transmitted. For example, the transmitter 40 may include a moreflexible material than the housing 201, so as to properly transmit atactile sense such as vibration, tapping, or rolling of the vibrator 203to the user.

The vibrator 203 may be disposed in the accommodation space of thehousing 201, and may be driven by a magnetic field generated by anelectrical signal applied to the coil 205. The vibrator 203 may includea material to be driven by the magnetic field. The vibrator 203 may beconstrued as a “magnetic circuit and mass body”.

For example, the vibrator 203 may be made of soft magnetic materialshaving intrinsic coercivities below at least 1000 amperes/meter (A/m),among ferromagnetic materials, and include a material having a structuresuch as steel, powder, alloy, alloy powder, composites, or ananostructure including at least one of elements such as Fe, Ni, Si, Mn,and Zn. The overall configuration may not need to be made of a singlematerial.

In another example, the vibrator 203 may include a material purelyincluding Cu or W having a specific gravity over 8, among paramagneticmaterials, or a material having a structure such as alloy, alloy powder,composites, or a nanostructure including at least one of the softmagnetic elements such as Fe, Ni, Si, Mn, and Zn mentioned above.Similarly, the material and the structure of the magnetic circuit andmass body may not need to be uniform.

A portion of the vibrator 203 may include a material having a structuresuch as steel, powder, alloy, alloy powder, composites, or ananostructure including at least one of elements such as Fe, Co, Ni, Nd,Ni, B, and Zn as the ferromagnetic materials, and include a materialmagnetized such that the N pole and the S pole thereof may bedistinguished in a vertical direction of FIG. 14.

The elastic member 204 may connect the housing 201 and the vibrator 203such that the vibrator 203 may vibrate with respect to the housing 201.For example, the elastic member 204 may include a low paramagnetic ordiamagnetic material, for example, stainless steel, plastic, or rubber,which has an elasticity that may deform by an external force and berestored to its original shape immediately when the external forcedisappears.

The elastic member 204 may include a fixture 204 a fixed to the housing201, a support 204 b configured to support the vibrator 203, and aconnector 204 c configured to connect the fixture 204 a and the support204 b. For example, a diameter of the fixture 204 a may be greater thana diameter of the support 204 b.

Meanwhile, although FIGS. 14 and 15 exemplarily illustrate a case inwhich the fixture 204 a and the support 204 b are ring-shaped, a support214 b of an elastic member 214 may include a plurality of segments, asshown in FIG. 16, which may also apply to a fixture 214 a.

The coil 205 may form a magnetic field to drive the vibrator 203 usingthe electrical signal applied thereto. For example, the coil 205 mayinclude a planar coil, a solenoid coil, or an electromagnetic coilhaving a core including soft magnetic materials.

FIG. 17 is a block diagram of a tactile actuator according to anembodiment.

Referring to FIG. 17, the haptic device 1 may include the tactileactuator 200, the database 10, the controller 20, the communicator 50, auser interface 60, and a sensor 70.

The user interface 60 may receive an instruction directly from the user.For example, the user interface 60 may be a keyboard, a mouse, or atouch panel. However, the type of the user interface 60 is not limitedthereto.

The sensor 70 may sense an external environment of the tactile actuator200. For example, the sensor 70 may sense temperature, humidity,pressure, or light intensity, convert the sensed information into anelectric signal, and transmit the electric signal to the controller 20.However, the type of the sensor 70 is not limited thereto.

The database 10 may store acoustic information. For example, in additionto a source in a form of a sound file such as MP3 which is generallyused, a text or an image including information related to notes, rests,and various signs modifying the same that may be written in a score suchas a five-line staff may be stored in the database 10. Data receivedfrom the user interface 60, the sensor 70, and/or the communicator 50may also be stored in the database 10. A plurality of preset drivingmodes may also be stored in the database 10.

The communicator 50 may receive acoustic information through wired orwireless communication with another communication device. For example,the communicator 50 may receive external data such as a sound, a score,an image, and a text through the Internet, and transmit the externaldata to the controller 20 and/or the database 10.

The user interface 60, the sensor 70, the database 10, and thecommunicator 50 may be collectively referred to as an “informationproviding device”. The information providing device may provide drivinginformation collected by the controller 20. An embodiment relates to thetactile actuator 200 that may operate in a plurality of driving modesbased on information provided from the information providing device tothe controller 20. However, the type of the collected information or thetype of the device providing the information is not limited thereto. Forexample, the driving information may include the acoustic informationdescribed above. That is, the driving information may include a text oran image including information related to notes, rests, time signatures,tempos, articulations, dynamic markings, and various signs modifying thesame that may be written in a score such as a five-line staff.

The controller 20 may determine one of the plurality of preset drivingmodes based on the collected driving information. Here, the drivinginformation collected by the controller 20 may be information receivedfrom the information providing device. The controller 20 may determine acharacteristic of an electrical signal to be applied to the coil 205based on the determined driving mode. Here, the characteristic of theelectrical signal may include an intensity, an apply time, a frequency,and a waveform.

An embodiment may enable vibration in a low frequency region by changinga physical property of the elastic member 204. Table 1 showing anelasticity coefficient of an elastic member induced from a mass of avibrator and a resonant frequency of an existing tactile actuator basedon the following Equation 1, and Table 2 showing an elasticitycoefficient of the elastic member 204 of the tactile actuator 200 aresuggested as follows.

$\begin{matrix}{f = {\frac{1}{2\pi}\sqrt{\frac{k}{m}}}} & \left\lbrack {{Equation}\mspace{14mu} 1} \right\rbrack\end{matrix}$

TABLE 1 Elasticity Coefficients of Elastic Members Used for ExistingTactile Actuator Spring Measured Mass Induced Spring No. Freq. (Hz) (g)Constant (N/mm) 1 154.7 1.578 1.491 2 154.1 1.578 1.479 3 152.7 1.5781.453 4 149.8 1.578 1.398 5 153.0 2.23 2.061 6 160.0 2.23 2.254

TABLE 2 Elasticity Coefficients of Elastic Members Used for TactileActuator According to Embodiment Spring Measured Mass Induced Spring No.Freq. (Hz) (g) Constant (N/mm) 7 98.7 0.65 0.250 8 81.4 0.79 0.207 975.7 0.93 0.210 10 85.3 1.09 0.313 11 78.2 1.04 0.251

Referring to Table 1 and Table 2, the elasticity coefficient of theelastic member 204 may be set to be over 0.2 newtons per millimeter(N/mm) and below 0.35 N/mm such that the tactile actuator 200 including,for example, the vibrator 203 with a mess ranging from 0.6 to 1.1 grams(g) may have a low resonant frequency below 100 Hz.

FIG. 18 is a graph conceptually illustrating a driving region withrespect to a frequency, in a tactile actuator according to anembodiment, and FIG. 19 is a graph illustrating a relationship betweenan actually measured frequency and an acceleration, in the tactileactuator according to an embodiment.

A solid line is a graph illustrating an operation of the tactileactuator 200, a dot-and-dash line is a graph illustrating an operationof an existing general linear resonant actuator (LRA), and a broken lineis a graph illustrating an operation of a multifunction vibrationactuator with an improved driving frequency band from the existinggeneral LRA.

Referring to FIGS. 18 and 19, the tactile actuator 200 may have at leasttwo driving modes based on the electrical signal applied to the coil205.

Referring to the graph (dot-and-dash line) of the existing general LRA,the existing general LRA has a maximum vibration force at a singleresonant frequency f_c above 170 Hz, and has a drivable frequency bandin a relatively narrow frequency band A3.

Since existing tactile methods are limited to vibration, outputs in afrequency band in which periodic vibration is not formed were defined asnoise and ignored, and thus the tactile methods failed to providevarious tactile senses.

Meanwhile, referring to the graph (solid line) of the tactile actuator,the tactile actuator has at least one resonant frequency f_a1 below 160Hz, and may suggest a tactile sense in a form of vibration that isoutput from an existing haptic device in a frequency band A11 includingthe corresponding resonant frequency f_a1.

Further, in a region below a threshold frequency f_a2 which isapproximately ⅓ the corresponding resonant frequency f_a1, the tactileactuator includes at least one different frequency band A12 in which aforce may be provided, rather than vibration, and the correspondingforce may be tactile senses such as tapping and rolling. Here, thethreshold frequency may be a minimum frequency at which periodicvibration is formed without showing collapse of a waveform generatedbased on an input electrical signal.

FIG. 20 is a graph illustrating a relationship between a measuredfrequency and an acceleration when a square wave electrical signalhaving a low frequency is applied, in a tactile actuator according to anembodiment, and FIG. 21 is a graph illustrating a relationship between ameasured frequency and an acceleration when a sine wave electricalsignal having a low frequency is applied, in the tactile actuatoraccording to an embodiment.

A solid line is a graph illustrating an operation of the tactileactuator 200, and a dot-and-dash line is a graph illustrating anoperation of the existing general LRA.

Referring to FIGS. 20 and 21, when an electrical signal of a lowfrequency flows, the existing general LRA showed noise unsuitable foractual use, whereas the tactile actuator 200 showed a vibration patternsuitable for a waveform of the provided external electrical signal.

FIG. 22 illustrates a control method for a tactile actuator according toan embodiment, FIG. 23 illustrates an example of an operation of thetactile actuator according to an embodiment, and FIG. 24 illustratesanother example of an operation of the tactile actuator according to anembodiment.

Referring to FIG. 22 through 24, driving information input through theinformation providing device 10, 50, 60, 70 of FIG. 17 may be collectedby the controller 20, in operation 300. Based on the driving informationcollected in operation 300, the controller 20 may determine anelectrical signal to be applied to the coil 205. The controller 20 maydetermine an apply time, an intensity, and/or a frequency of theelectrical signal. The controller 20 may determine one of a plurality ofpreset driving modes, and determine the frequency of the electricalsignal based on the determined driving mode, in operation 310. Here, theplurality of preset driving modes may include, for example, a generalvibration mode, a tapping mode, and/or a rolling mode. Hereinafter, acase in which a first set mode is the general vibration mode, a secondset mode is the tapping mode, and a third set mode is the rolling modewill be described.

For example, the driving information collected in operation 300 mayinclude acoustic information related to duration information of a sound,pitch information of the sound, whether a dynamic marking is present,and a type of the dynamic marking. In operation 310, the controller 20may determine the apply time of the electrical signal based on theduration information of the sound, determine the intensity of theelectrical signal based on the pitch information of the sound, determinethe driving mode to be a general vibration mode, a tapping mode, or arolling mode based on whether a dynamic marking is present, whether thedynamic marking is a forte-type marking, or whether the dynamic markingis a piano-type marking, and determine the frequency of the electricalsignal to correspond to the determined driving mode.

Whether the driving mode determined in operation 310 is the generalvibration mode may be determined, in operation 320. When the determineddriving mode is the first set mode (general vibration mode), thecontroller 20 may determine the frequency of the electrical signal to bea first set frequency f_H higher than a threshold frequency, which is aminimum frequency to form a vibration force with a shape of a periodicsine wave, in operation 330. The controller 20 may apply the electricalsignal corresponding to the determined apply time, the determinedintensity, and the determined frequency to the coil 205, in operation380. The first set frequency f_H may be set to be a value belonging tothe frequency band A11 of FIG. 18 around the resonant frequency of thetactile actuator 200. For example, the first set frequency f_H may be avalue below 160 Hz.

When the electrical signal having the first set frequency f_H is appliedin operation 380, the vibrator 203 may vibrate up and down in theaccommodation space of the housing 201, as shown in FIG. 23. Thevibration may be transmitted to the user sequentially through theelastic member 204, the housing 201, and the transmitter 40. In thefirst set mode, a frequency high enough to form a periodic vibrationforce may be input. Thus, similar vibration may be generated withoutbeing affected greatly by a type of an input waveform. That is, the typeof the input waveform in the first set mode is not limited.

Meanwhile, when the driving mode determined in operation 310 is a setmode other than the first set mode, the controller 20 may determine thefrequency of the electrical signal to be the second set frequency f_Lwhich is lower than the first set frequency f_H, in operation 340. Thesecond set frequency f_L may be determined to be a value lower than thethreshold frequency. For example, the second frequency f_L may be avalue below ⅓ the resonant frequency of the tactile actuator 200.

After operation 340 is performed, the controller 20 may determinewhether the driving mode is the second set mode (tapping mode), inoperation 350. When the driving mode is determined to be the second setmode (tapping mode) in operation 350, the controller 20 may determine awaveform of the electrical signal to be a square wave or a pulse wave.Conversely, when the driving mode is determined to be the third set mode(rolling mode) in operation 350, the controller 20 may determine thewaveform of the electrical signal to be a sine wave. The controller 20may apply the electrical signal of the set frequency and the setwaveform to the coil 205, in operation 380.

When the electrical signal having the second set frequency f_L isapplied in operation 380, the vibrator 203 may not form a periodicvibration force, and thus transmit a different tactile sense to the userbased on the input waveform, as described below.

First, in a case in which the waveform of the input electrical signal isa sine wave, the vibrator 203 not forming a periodic vibration force maymove up and down aperiodically. In addition, due to a characteristic ofthe sine wave, an intensity of the electrical signal input into the coil205 may change gently. Thus, the user may feel a tactile sense ofrolling through the above motion. Herein, “rolling” may be construed ascollectively referring to a series of aperiodic tactile senses. Whenapplying the above conditions to a prototype in practice, the user felta tactile sense of rolling.

Next, in a case in which the waveform of the input electrical signal isa square wave or a pulse wave, the vibrator 203 not forming a periodicvibration force may similarly move up and down aperiodically. However,due to a characteristic of the square wave or the pulse wave, theintensity of the electrical signal input into the coil 205 may changeradically. Thus, at each periodic instant at which the intensity of theelectrical signal changes, an acceleration in a direction in which thevibrator 203 moves up and down may change much greatly, when compared toother sections. A tactile sense that the user feels at an instant atwhich the intensity of the electrical signal changes may increase athreshold value of a sense of touch of the user, which may cause asensory adaptation such that the user may not feel a tactile sense inremaining sections. Thus, the user may feel a tactile sense of“tapping”. Herein, “tapping” may be construed as collectively referringto a tactile sense of periodically repeating an impulse high enough tobe more distinguishing than the remaining sections. When applying theabove conditions to a prototype in practice, the user felt a tactilesense of tapping.

That is, when an electrical signal having a frequency below ⅓ theresonant frequency of the tactile actuator 200 is input, the user mayfeel at least two different tactile senses based on a waveform of theelectrical signal.

Meanwhile, for example, in a case in which a distance between thevibrator 203 and the transmitter 40 is sufficiently close, or asufficient voltage is input, the vibrator 203 may be in direct contactwith the transmitter 40, as shown in FIG. 24, thereby transmitting aforce directly to the user through the transmitter 40.

Hereinafter, graphs showing experiment results using the tactileactuator 200 will be described in detail.

FIG. 25 is a graph illustrating a change in an acceleration with respectto a change in an intensity of a square wave input electrical signal of5 hertz (Hz), in tactile actuators having different resonantfrequencies, and FIG. 26 is a graph illustrating a change in anacceleration with respect to a change in a frequency of a square waveinput electrical signal of 90 milliamperes (mA), in the tactileactuators having different resonant frequencies.

Through the experiments, it was learned that a user may feel a tactilesense of tapping when the vibrator 203 operates with an acceleration of0.2 G. Referring to FIGS. 25 and 26, in a case in which the resonantfrequency of the tactile actuator 200 is below 160 Hz, the vibrator 203may operate with an acceleration over 0.2 G although an electricalsignal with an intensity of 90 mA and a frequency of 5 Hz is applied.Conversely, in a case in which the resonant frequency of the tactileactuator is 180 Hz which is a bit greater than 160 Hz, an electricalsignal over 130 mA which is about 1.5 times 90 mA may need to be appliedsuch that the vibrator 203 may operate with an acceleration over 0.2 G.

In a case in which a mass of the vibrator 203 is below 2 g, the tactileactuator 200 may set an elasticity coefficient of the elastic member 204to be below 2.021 N/mm, thereby setting the resonant frequency to bebelow 160 Hz. Meanwhile, in a case in which the mass of the vibrator 203is over 2 g, the tactile actuator 200 may set the elasticity coefficientof the elastic member 204 to be over 2.021 N/mm, thereby setting theresonant frequency to be below 160 Hz.

FIGS. 27 through 30 illustrate waveforms of vibrators exhibited inresponse to a change in a square wave electrical signal input intotactile actuators having resonant frequency characteristics of 80 Hz,120 Hz, 160 Hz, and 180 Hz, respectively, and FIG. 31 is a graphillustrating threshold frequencies of tapping and vibration when asquare wave electrical signal is applied, in the tactile actuatorshaving different resonant frequencies.

Referring to FIGS. 27 through 30, when a square wave electrical signalover a predetermined frequency is applied, the vibrator 203 may form avibration force of a shape of a sine wave which is a periodic waveform,as shown in the graphs in the right column of each drawing. Thus, underthe above conditions, the tactile actuator may provide a tactile senseof “vibration” to the user.

Conversely, as shown in the graphs in the left column of each drawing,the vibrator 203 may not form a periodic vibration force in a regionbelow the predetermined frequency, and the graphs partially collapse.However, due to a characteristic of the square wave, at each periodicinstance at which the intensity of the electrical signal changes, anacceleration of the vibrator may change much greatly, when compared toother sections. Thus, under the above conditions, the tactile actuator200 may provide a tactile sense of “tapping” to the user.

As described above, the tactile sense that the tactile actuator 200provides to the user may be divided as vibration or tapping based on thepredetermined frequency. The predetermined frequency may also bereferred to as a threshold frequency or a division frequency.

Referring to FIGS. 27 through 30, as the resonant frequency of thetactile actuator 200 increases, the threshold frequency may alsoincrease, which is shown in FIG. 31. In a control method for the tactileactuator 200, the first set frequency f_H and the second set frequencyf_L may be set based on the threshold frequency of FIG. 31.

FIGS. 32 through 35 illustrate waveforms of vibrators exhibited inresponse to a change in a pulse wave electrical signal input intotactile actuators having resonant frequency characteristics of 80 Hz,120 Hz, 160 Hz, and 180 Hz, respectively, and FIG. 36 is a graphillustrating threshold frequencies of tapping and vibration when a pulsewave electrical signal is applied, in the tactile actuators havingdifferent resonant frequencies.

Referring to FIGS. 32 through 35, when a pulse wave electrical signal isinput, a vibrator may have a similar waveform to a case in which asquare wave electrical signal is input. Thus, when a pulse waveelectrical signal below a threshold frequency is applied to the tactileactuator, the tactile actuator may provide a tactile sense of “tapping”to the user. When a pulse wave electrical signal over the thresholdfrequency is applied to the tactile actuator, the tactile actuator mayprovide a tactile sense of “vibration” to the user.

Referring to FIGS. 32 through 35, as the resonant frequency of thetactile actuator increases, the threshold frequency may also increase,which is shown in FIG. 36.

Meanwhile, with respect to the tactile actuator having the same resonantfrequency, a threshold frequency when inputting a pulse wave electricalsignal may be about two times a threshold frequency when inputting asquare wave electrical signal.

In a control method for the tactile actuator, the first set frequencyf_H and the second set frequency f_L may be set based on the thresholdfrequency of FIG. 36.

FIGS. 37 through 40 illustrate waveforms of vibrators exhibited inresponse to a change in a sine wave electrical signal input into tactileactuators having resonant frequency characteristics of 80 Hz, 120 Hz,160 Hz, and 180 Hz, respectively, and FIG. 41 is a graph illustratingthreshold frequencies of rolling and vibration when a sine waveelectrical signal is applied, in the tactile actuators having differentresonant frequencies.

Referring to FIGS. 37 through 40, when a sine wave electrical signalover a predetermined frequency is applied, a vibrator may form avibration force of a shape of a sine wave which is a periodic waveform,as shown in the graphs in the right column of each drawing. Thus, underthe above conditions, the tactile actuator may provide a tactile senseof “vibration” to the user.

Conversely, as shown in the graphs in the left column of each drawing,the vibrator may not form a periodic vibration force in a region belowthe predetermined frequency, and the graphs partially collapse. Thevibrator not forming a periodic vibration force may have an accelerationof a vertical motion aperiodically. Meanwhile, due to a characteristicof the sine wave, an intensity of the electrical signal may changegently, and thus the user may feel a tactile sense of “rolling” throughthe above motion.

As described above, the tactile sense that the tactile actuator providesto the user may be divided as vibration or rolling based on thepredetermined frequency.

Referring to FIGS. 37 through 40, as the resonant frequency of thetactile actuator increases, the threshold frequency may also increase,which is shown in FIG. 41. In a control method for the tactile actuator,the first set frequency f_H and the second set frequency f_L may be setbased on the threshold frequency of FIG. 41.

FIG. 42 illustrates a control method for a tactile actuator according toanother embodiment. Unless otherwise disclosed, the description of thecontrol method for the tactile actuator provided with reference to FIG.22 may also apply to the other embodiment.

Referring to FIG. 42, in a control method for the tactile actuator 200,when a driving mode is a general vibration mode in operation 320, thecontroller 20 may determine a frequency of an electrical signal to beapplied to be a first set frequency f_H. With reference to FIG. 31, 36,or 41, the first set frequency f_H may be set to be a value greater thana first threshold frequency which is a minimum frequency to provide atactile sense of “vibration” under provide conditions.

When the driving mode is not the general vibration mode in operation320, the controller 20 may determine whether the driving mode is atapping mode, in operation 350.

In a case in which the driving mode is the tapping mode in operation350, the controller may determine the frequency of the electrical signalto be applied to be a second set frequency f_L1, in operation 351. Withreference to FIG. 31 or 36, the second set frequency f_L1 may be set tobe a value less than a second threshold frequency which is a maximumfrequency to provide a tactile sense of “tapping” under providedconditions.

In a case in which the driving mode is not the tapping mode in operation350, the controller 20 may determine the frequency of the electricalsignal to be applied to be a third set frequency f_L2, in operation 352.With reference to FIG. 41, the third set frequency f_L2 may be set to bea value less than a third threshold frequency which is a maximumfrequency to provide a tactile sense of “rolling” under providedconditions.

Meanwhile, as shown in FIGS. 31, 36, and 41, the third thresholdfrequency which is a maximum frequency to provide a tactile sense of“rolling” may be greater than the second threshold frequency which is amaximum frequency to provide a tactile sense of “tapping” under the sameconditions. Thus, the third set frequency f_L2 may be set to be higherthan the second set frequency f_L1. Meanwhile, the first set frequencyf_H may be set to be a value greater than the second set frequency f_L1and the third set frequency f_L2. That is, the third set frequency f_L2may be greater than the second set frequency f_L1, and the first setfrequency f_H may be greater than the third set frequency f_L2.

FIG. 43 illustrates driving modes in which a tactile actuator operatesbased on types of dynamic markings according to an embodiment.

Referring to FIG. 43, the controller 20 of the haptic device 1 mayinterpret a meaning of a dynamic marking of acoustic information, andcontrol a driving mode of the driver 30 based on a type of the dynamicmarking.

For example, in a case in which the dynamic marking is a forte-typemarking, the controller 20 may operate the tactile actuator 200 in ageneral vibration mode. Meanwhile, in a case in which the dynamicmarking is a piano-type marking, the controller 20 may operate thetactile actuator 200 in a tapping mode or a rolling mode. However, theembodiments are note limited thereto. A point to switch the driving modebased on the dynamic marking may be set by a user at random. Any exampleof providing a different driving mode based on whether a dynamic markingis present and a type of the dynamic marking should be construed asfalling within the scope of the present invention.

By the above configuration, the user may be provided with a differenttactile sense based on whether a dynamic marking is present and a typeof the dynamic marking, thereby recognizing a change in the acousticinformation through the tactile sense.

Meanwhile, a case in which a criterion to distinguish a driving mode isa dynamic marking is exemplarily suggested above. However, the drivingmode may also be set differently based on a different type of acousticinformation. For example, the controller 20 may interpret a meaning of aslur of the acoustic information, and operate the tactile actuator 200in the rolling mode in a section in which notes are linked by the slur.Thus, a musical feeling of softly slurring the notes linked by the slurmay be expressed as a tactile sense.

The methods according to the above-described embodiments may be recordedin non-transitory computer-readable media including program instructionsto implement various operations of the above-described embodiments. Themedia may also include, alone or in combination with the programinstructions, data files, data structures, and the like. The programinstructions recorded on the media may be those specially designed andconstructed for the purposes of embodiments, or they may be of the kindwell-known and available to those having skill in the computer softwarearts. Examples of non-transitory computer-readable media includemagnetic media such as hard disks, floppy disks, and magnetic tape;optical media such as CD-ROM discs, DVDs, and/or Blue-ray discs;magneto-optical media such as optical discs; and hardware devices thatare specially configured to store and perform program instructions, suchas read-only memory (ROM), random access memory (RAM), flash memory(e.g., USB flash drives, memory cards, memory sticks, etc.), and thelike. Examples of program instructions include both machine code, suchas produced by a compiler, and files containing higher level code thatmay be executed by the computer using an interpreter. Theabove-described devices may be configured to act as one or more softwaremodules in order to perform the operations of the above-describedembodiments, or vice versa.

A number of embodiments have been described above. Nevertheless, itshould be understood that various modifications may be made to theseembodiments. For example, suitable results may be achieved if thedescribed techniques are performed in a different order and/or ifcomponents in a described system, architecture, device, or circuit arecombined in a different manner and/or replaced or supplemented by othercomponents or their equivalents.

Accordingly, other implementations are within the scope of the followingclaims.

The invention claimed is:
 1. A haptic device, comprising: a database configured to store acoustic information or receive the acoustic information from an external device; a controller configured to convert the acoustic information into an electrical signal corresponding to a predetermined pattern; and a tactile actuator configured to provide a user with a patterned tactile signal, wherein the tactile actuator comprises: a driver configured to generate a motion signal based on the electrical signal; and a transmitter configured to transmit the patterned tactile signal to the user using the motion signal, wherein the controller is configured to determine one of a plurality of predetermined driving modes based on the acoustic information, and convert the acoustic information into the electrical signal having a frequency corresponding to the determined driving mode, and wherein the controller is configured to: determine a frequency of the electrical signal to be a first set frequency when the driving mode is a first set mode, and determine the frequency of the electrical signal to be a second set frequency when the driving mode is a set mode other than the first set mode, the second set frequency being lower than the first set frequency.
 2. The haptic device of claim 1, wherein the controller is configured to: determine a waveform of the electrical signal to be a square wave or a pulse wave when the driving mode is a second set mode, and determine the waveform of the electrical signal to be a sine wave when the driving mode is a third set mode.
 3. The haptic device of claim 1, wherein the driving mode includes a general vibration mode, a tapping mode, and a rolling mode.
 4. The haptic device of claim 3, wherein the controller is configured to: determine a frequency of the electrical signal to be a first set frequency when the driving mode is the general vibration mode, determine the frequency of the electrical signal to be a second set frequency when the driving mode is the tapping mode, the second set frequency being lower than the first set frequency, and determine the frequency of the electrical signal to be a third set frequency when the driving mode is the rolling mode, the third set frequency being higher than the second set frequency and lower than the first set frequency.
 5. The haptic device of claim 1, further comprising: an information providing device configured to provide information collected by the controller, wherein the information providing device comprises at least one of a user interface configured to receive an instruction of a user, a sensor configured to sense an external environment, a memory configured to store data, and a communicator configured to receive information through communication with another communication device. 